Least Absolute Shrinkage is Equivalent to Quadratic Penalization
نویسنده
چکیده
Adaptive ridge is a special form of ridge regression, balancing the quadratic penalization on each parameter of the model. This paper shows the equivalence between adaptive ridge and lasso (least absolute shrinkage and selection operator). This equivalence states that both procedures produce the same estimate. Least absolute shrinkage can thus be viewed as a particular quadratic penalization. From this observation, we derive an EM algorithm to compute the lasso solution. We finally present a series of applications of this type of algorithm in regression problems: kernel regression, additive modeling and neural net training.
منابع مشابه
Outcomes of the Equivalence of Adaptive Ridge with Least Absolute Shrinkage
Adaptive Ridge is a special form of Ridge regression, balancing the quadratic penalization on each parameter of the model. It was shown to be equivalent to Lasso (least absolute shrinkage and selection operator), in the sense that both procedures produce the same estimate. Lasso can thus be viewed as a particular quadratic penalizer. From this observation, we derive a fixed point algorithm to c...
متن کاملLasso and equivalent quadratic penalized models
The least absolute shrinkage and selection operator (lasso) and ridge regression produce usually different estimates although input, loss function and parameterization of the penalty are identical. In this paper we look for ridge and lasso models with identical solution set. It turns out, that the lasso model with shrink vector λ and a quadratic penalized model with shrink matrix as outer produ...
متن کاملShrinkage simplex-centroid designs for a quadratic mixture model
A simplex-centroid design for q mixture components comprises of all possible subsets of the q components, present in equal proportions. The design does not contain full mixture blends except the overall centroid. In real-life situations, all mixture blends comprise of at least a minimum proportion of each component. Here, we introduce simplex-centroid designs which contain complete blend...
متن کاملRobust high-dimensional semiparametric regression using optimized differencing method applied to the vitamin B2 production data
Background and purpose: By evolving science, knowledge, and technology, we deal with high-dimensional data in which the number of predictors may considerably exceed the sample size. The main problems with high-dimensional data are the estimation of the coefficients and interpretation. For high-dimension problems, classical methods are not reliable because of a large number of predictor variable...
متن کاملAugmented Lagrangian method for solving absolute value equation and its application in two-point boundary value problems
One of the most important topic that consider in recent years by researcher is absolute value equation (AVE). The absolute value equation seems to be a useful tool in optimization since it subsumes the linear complementarity problem and thus also linear programming and convex quadratic programming. This paper introduce a new method for solving absolute value equation. To do this, we transform a...
متن کامل